7.1.1 Peripheral Research & Design Settings

[TODO Move 3.4.3 etc. to here and remove all refs to 3.4.3]

The majority of examples and learnings shared in this chapter come from my participation as an expert researcher and designer in two industrial research projects:

1. BBC R&D’s Cornmarket Project, which explored through user experience design, technical prototyping and participatory research, how individuals might interact with data through a Personal Data Store interface (see 3.4.3.3)
2. Sitra/Hestia.ai’s #digipower Project, a successor to Case Study Two, in which European politicians examined companies’ data practices through exercising data rights and conducting technical audits (see 3.4.3.4) [TODO add links to both the page, and the sitra report, and (Bowyer, Pidoux, et al., 2022; Pidoux et al., 2022)]

In addition, my participation as an interface designer and front-end software developer in the following two academic research projects contributes secondarily to this chapter:

3. Connected Health Cities (CHC)’s SILVER Project, where I, along with a backend developer and a team of researchers, developed a prototype health data viewing interface for Early Help support workers (see 3.4.3.1).
4. Digital Economy Research Centre (DERC)’s Healthy Eating Web Augmentation Project, which explored the use of web augmentation techniques to modify the user interface of takeaway service Just Eat to insert health information, in support of healthy eating (see 3.4.3.2).

7.1.2 Attribution of Insights

While this thesis is my own original work, and many ideas presented in this chapter are fully original, some of the specific details, theories and ideas presented in this chapter arose or were developed or augmented through my close collaboration, discussion and ideation with other researchers, including:

• Jasmine Cox, Suzanne Clarke, Tim Broom, Rhianne Jones, Alex Ballantyne and others at BBC R&D;
• Paul-Olivier Dehaye, Jessica Pidoux, Francois Quellec at Hestia.ai;
• Stuart Wheater of Arjuna Technologies and Kyle Montague of Open Lab during the SILVER project; and
• Louis Goffe of Open Lab on the DERC Healthy Eating project
• earlier innovation work with Alistair Croll at Rednod, Montréal, Canada (circa 2011) and with Megan Beynon at IBM Hursley, UK (circa 2006).

Due to these collaborations and the ongoing and parallel nature of many of these projects to my PhD research, it is impossible to precisely delineate the origin of each idea or insight. In practice, ideas from my developing thesis and own thinking informed the projects’ trajectories and thinking, and vice-versa. These ideas would not have emerged in this form without my participation, so they are not the sole intellectual property of others, but equally I would not have reached the same conclusions alone, so the ideas are not solely my own either. All diagrams and illustrations were produced by me, except where specified, and the overall synthesis and framing presented in this chapter is my own original work. Where this chapter includes material from the four projects, that material is either already public, or permission has been obtained from the corresponding project teams.

7.2.1 The Role of Personal Data

In the modern world, where almost anything can be encoded as data, and given many previously analogue objects and activities now have digital equivalents, the concept of data has become broad and hard to pin down. Underlying Human Data Relations is to explain what roles data can play in people’s lives – what it is to people. Through the Case Studies, external work and my prior learning, I have so far identified 8 distinct lenses to consider how people might relate to it. These are modelled in Table 15.

When considering HDR, it is important to recognise that people may think of their personal data through any or all of these ‘lenses’ [Karger et al. (2005);2.2.2] at any given time, and any process or system design involving data interaction should take these into account.

Looking across this set of lenses, it is possible to identify four specific roles that data can serve:

1. Data has a role as an artifact of value to your life;
2. Data has a role in informing you about yourself, the world, and the prior or recent actions of others that may affect you;
3. Data has a role as a usable material with which to effect change in your life;
4. Data has a role as a means to monitor changes in data holders’ behaviours, digital influences upon you or changes within your life.

7.2.2 Human Data Interaction or Human Information Interaction?

To unpack HDR further, it is important to highlight the difference between humans relating to data, and humans relating to information. Human Data Interaction (HDI) concerns the way people interact with data. Mortier et al. (Mortier et al., 2013, 2014) defined the field of HDI without distinguishing data (the digital artifact stored on computer) from information (the facts or assertions that said data can provide when interpreted). This is an important distinction. The parallel field of Human Information Interaction (HII) originated in library sciences, and considers the way humans relate to information without regard to the technologies involved (Marchionini, 2008). William Jones et al. called for a new sub-field of HII in an HCI context², observing that it is important to include a focus on information interaction because HCI can “unduly focus attention on the computer when, for most people, the computer is a means to an end – the effective use of information” (Jones et al., 2006). DIKW theory [2.1] highlights that interpretation of data to obtain information is a discrete activity. This was borne out in the findings of Case Study Two, where it became clear that participants have distinct needs from data, and from information (5.4.3.2). Access to data and information is critical to both understanding and useability, as detailed in section 6.1.2 and 6.1.3.

Drawing on this theory, we can see then that in considering Human Data Relations, there are in fact three distinct artifacts to consider:

1. data - the stored digital artifacts pertaining to users held by organisations for algorithmic processing and human reference, copies of which can be obtained using individual data rights.
2. information about individuals - the collection of facts and assertions about the individual and their life, which are obtained through human or algorithmic interpretation of stored data (or in some organisations’ case, through analytical inference).
3. information about data (also categorised in Table 9 / 5.3.1 as metadata) - stored facts about the data, such as where it has been stored, who has accessed it, how it was collected, what it means, or when it has been shared externally.

7.2.3 The Two Distinct Motivations for Human Data Relations

By making this distinction between the two types of information which people might interact with, and considering the six wants in Chapter 6, it becomes clear that there are two very different reasons why people might want better HDR:

1. to acquire information about one’s data, so that one might exert control over and make informed choices about where the data is held and how it is used, in order to be treated fairly and gain more control over the use of one’s personal data. This is Personal Data Ecosystem Control (PDEC).

2. to acquire information about oneself, so that one might gain insights into one’s own behaviour and gain personal benefits from those insights or them to make changes in one’s life. This is Life Information Utilisation (LIU).

The two distinct processes that individuals might go through in pursuit of these motives are exemplified in Figure 30. PDEC is a process of holding organisations to account over and managing what happens to personal data, often regardless of what it means, whereas LIU is more concerned with what the data means and its inherent value as encoded life information, regardless of where it is stored and how it is used³. This novel way of modelling the motivations for data interaction were first proposed in my 2021 workshop paper (Bowyer, 2021).

7.2.4 Four Objectives for Human Data Relations

In order to provide value to future researchers, activists and innovators, this chapter contributes a map of the HDR opportunity landscape. This map is expressed in two parts across this section and 7.4. As a first step, we can take the “six wants from data relations” Chapter 6, and map reduce those to four simple ‘landscape objectives’ which shape our ultimate goals for effective HDR in this landscape of opportunity:

1. Data Awareness & Understanding;
2. Data Useability¹⁵;
3. Ecosystem Awareness & Understanding and
4. Ecosystem Negotiability.

As Figure X shows, the need for data to be understandable, visible and useable, applies to all types of data, whether that data is interpretable as life information (information within the data, that says something about the individual) or ecosystem information (information about the data, where it is held and how it is used). These two types of information will collectively be referred to as human information, and will be used in describing the HDR landscape in subsequent sections.

7.2.5 Better Human Data Relations as a Recursive Public

Before engaging with the practicalities of pursuing these HDR objectives, it is valuable to revisit the stance from which we approach this change. As outlined in 3.2, the research of this PhD has been grounded in participatory action research and experience-centred design; by using a Digital Civics (Vlachokyriakos et al., 2016) frame to gain deep understanding of people’s needs and the ways those needs are not fully met, we can interpret and model how the world needs to change. Section 3.2 already outlined that we can consider such research as political, seeking to correct an imbalance in the world. In this chapter, we look beyond identifying what change is needed, and step into the role of activist, exploring how individuals and groups can actually change the world they inhabit.

In doing so, we can consider ourselves (those who pursue better Human Data Relations, or HDR reformers as a shorthand) as a recursive public (Kelty, 2008; ‘Recursive Public (Discussion Page)’, no date), albeit a nascent one. This is a term originating in the free software movement to describe a “collective, independent of other forms of constituted power, capable of speaking to existing forms of power through the production of actually existing alternatives”. This term captures the idea that through various means at our disposal: participatory research, experience-centred design, engineering software prototypes, exertion of legal rights, and efforts to raise public awareness, we seek to modify the systems and practices we live within in pursuit of our goals. This idea of reconfiguring society in this way has been conceived as civic hacking (Crabtree, 2007; Levitas, 2013; Tauberer, 2014). The collective around better Human Data Relations does not yet exist as a named and identifiable public (Le Dantec, 2016) but its members congregate around emergent collectives in interconnected and overlapping spaces, most notably the MyData community (MyData, 2017) and its members, but also research and activism agendas including but not limited to: personal data lockers (CitizenMe, 2021; ‘Digi.me’, no date); digital rights (‘Open rights group: Who we are’, no date), gig economy worker rights (Kirven, 2018; ‘Worker info exchange’, 2022), privacy by design (Cavoukian, 2010), privacy activism (Davies, 1990; ‘Bits of freedom: Our focus’, 2000), data justice (Taylor, 2017; Crivellaro et al., 2019), critical algorithm studies (Gillespie and Seaver, 2016), humane technology (Harris, 2013) and explainable AI (‘Explainable AI: Making machines understandable for humans’, no date).

Whether these disparate groups coalesce into a single identifiable public remains to be seen, and so too whether the term this thesis offers of Human Data Relations is sufficient to capture that public. At the least, it provides a descriptive umbrella term. Nonetheless, the breadth of research and innovation and activism happening in this space validates both the need and the desire for such a recursive public around better HDR to exist – in fact, it already does, whether we name it such or not. Therefore, this chapter takes an unashamedly critical view of the status quo, favouring disruptive societal changes that would further the objectives of better Human Data Relations and providing actionable approaches that will be of use to the members of this public. The chapter asks, “How can we change the world into the one we want?”

7.3.1 Obstacles to the HDR Objective of Data Awareness & Understanding

In pursuit of visible, understandable data [6.1.1; 6.1.2], the first obstacle encountered is that in today’s complex digital landscape, most personal data is invisible, inaccessible or unrelatable. It is trapped in service providers’ databases, or on different devices or hard drives, or inaccessible due to software limitations or proprietary file formats (Abiteboul, André and Kaplan, 2015; Bowyer, 2018), or in formats. Participants of both Case Studies talked of ‘not knowing’ what data exists and of being ‘in the dark’. As Case Study Two showed, even where data is accessible, it is not relatable (‘legible’ (Mortier et al., 2014); 2.3.2). Thus the objective here is to tackle this obstacle and ensure that people not only have awareness of their data, but can understand (‘make sense’ [Gurstein (2011); 2.1.4]) of what it means.

[TODO make this an inset box not a table] [TODO Fix non showing caption]

Another important obstacle to consider here is what I call the Personal Data Diaspora⁵. As illustrated by Imogen Heap’s quote opening Chapter 1, an individual’s personal data is typically very widely dispersed. For example, if I consider just my movement tracking data, I have over time accumulated activity logs from walking, running, cycling, and driving which are stored by Nike+, MyFitnessPal, Strava, Google Fit, Fitbit, Apple Health and Google Maps, not to mention the records remaining on my different smart watches, smartphones and hard drives. This is the problem of Integration (Li, Forlizzi and Dey, 2010) that SI enthusiasts face [2.2.3]. Even aside from the issues this creates in terms of managing one’s data ecosystem [2.2.4], it means that it is impossible to view the history of my physical activity side by side, to spot patterns over time or make comparisons. To overcome this obstacle, approaches to data interfaces and life information modelling must be identified that recognise the scattered, complex reality of each individual’s personal data ecosystem and begin to make it visible and understandable. This is explored further in 7.3.3 and 7.3.4 below.

The takeaway for this HDR objective is that data awareness and understanding is a problem of representation. Invisible data should be visibly represented, and all data should be represented in the context of its interpretation as life information.

7.3.2 Obstacles to the HDR Objective of Data Useability¹⁵

To consider how to improve the useability of data, we must first consider what properties of data, as it typically exists today, make it hard to use. The primary obstacles are that most personal data is immobile, inaccessible, unmalleable and not interrogable.

It is immobile, in that it is very difficult to move a dataset out of the environment where it exists: most data exists in organisations’ internal databases, where it is tightly coupled to technology stacks, interfaces and business processes that use it. Separating one’s data from the service that holds the data is difficult and often impossible.

[TODO possibly move this paragraph elsewhere to avoid repetition with previous section] This setting of personal data also explains why it is inaccessible to individuals (in the sense of ‘effective access’ (Gurstein, 2011)). Data access requests such as GDPR are typically satisfied by creating a copy of the data, which creates problems of delay, divergence and understanding. Even then, as Case Study Two showed, this is incomplete [5.4.2.2] and much of the data is never made available. Its accessibility is also hindered by the technical nature of data. For organisational efficiency, data will often be stored in complex proprietary structures which are designed for the algorithmic efficiency of the specific operations the service provider wants to perform, rather than for general-purpose re-use.

Evident from individuals’ goals for use of their data [Table 12] is that people need to be able to ask questions of their data. This highlights the problem that data is not interrogable. It must stand for itself, and there is no obvious way to ask a question about the meaning of the data or about what the data says about a particular question, without either the co-operation of the data holder, or advanced technical skills in querying and data analysis. To be able to ask questions of data, it needs to be malleable - one needs the ability to break it down, look at it from different perspectives, and reconstitute it in different ways. This requires more than just an ability to produce visual representations of the data, but an ability to interact with the data and produce new interpretations and insights that can help to answer specific questions.

To overcome these obstacles, we need to find ways to extract data from its current constraints into environments where it can move freely and be examined and reconstituted without restriction.

To address these obstacles, the following insights could help:

[TODO make this an inset box not a table, image appears in box]

So, for data to be useable, we must change its nature. We have been trained by the computers that have existed up to now that the basic units for interacting with computer systems are files - these are the material of today’s personal computers. Where we do interact with data as information instead of files, that information is typically presented in limited contexts within certain products or apps. In line with the goal to move up the DIKW pyramid [2.1], we need smarter computer systems, that move beyond files (Bowyer, 2011) - systems whose basic units of interaction are pieces of human information. We need a human information operating system.

7.3.3 Obstacles to the HDR Objective of Ecosystem Awareness & Understanding

As I have established in 2.2.5, 2.3, 6.2 and 7.2, human data relations cannot be made effective without a sea change in the way that individuals are able to interact with the complex ecosystem of personal data that we each inhabit. Our personal data ecosystems are incredibly complex and largely invisible. For example, it is very easy to allow a handful of communication and social media apps access to your address book or contact list, and before you know it you have created a complex and unmanageable network of connections that silently sync and propogate your addresses and phone numbers across the Internet. And there are deeper layers which are not even slightly visible to users: networks of data brokers, advertisers and digital cookie companies exchange user identifiers, activity data and personal information about you while you browse or use apps (Pidoux et al., 2022). As the Case Studies showed, the ability to to build up a meaningful picture of your personal data ecosystem is completely absent [4.3.4.1] or severely limited, causing people to remain ‘in the dark’ and leads to feelings of fear (Bowyer et al., 2018), overload [2.2.4] and resignation [5.4.4.1]. Managing one’s personal data ecosystem is an overwhelming, unmanageable task that even personal data experts are not fully able to get a handle on. We do not feel ‘in control’ [Teevan (2001); 2.2.2]. The ability to provide a user with ecosystem transparency is hindered by the complexity and multiplicity of the data relationships they have been encouraged to set up, and by a lack of tools to provide a meaningful, or indeed any, view of those relationships. A further aspect to this obstacle is that in both Case Study contexts, no one individual or organisation has the ability to see the whole of a user’s data ecosystem [4.3.4.3; Cornford, Baines and Wilson (2013)], and there is little commercial motive to try and solve this problem, as every provider focuses just on their own apps, websites and services. Making one’s ecosystem visible, transparent and understandable is therefore an essential objective for better HDR.

Once we begin to think about storing and representing human information in ways that go beyond simply representing the information that is encoded within the data, and into the realm of what the data is about, new possibilities are unlocked. We can envisage building a PDS-type system that is not only a repository of personal data, but (thanks to proxy representations), a collection of ecosystem information and contextually-situated life information too, including information about relationships with data holders or other entities. This, however, exposes a secondary problem that any builder of such a system would face: a lack of metadata (as discussed in 2.2.2). Typically, much of the information stored on our hard drives lacks context about where it has come from, and how it relates to the individual in a holistic life/ecosystem sense. Where data access rights are executed (or data is shared via human means such as in 4.3.2.2), the attention is on the data itself: what it says. Case Study Two showed that some of the most desired information was not the data itself, but how it is used and shared and what is inferred from it (i.e. metadata [Table 9]), yet this was rarely forthcoming [Table 10]. There are many facets that can be quantified and recorded about a datapoint or dataset, as illustrated in Figure X, which I created at BBC R&D:

These facets can be mapped back to the 5 W’s that collectively make up the user’s context [Abowd and Mynatt (2000); #2.2.2-contextual]. It is notable that many of these facets are not explicitly recorded today, or would take significant work to capture; nonetheless, this exploration can serve as a useful reference for how information can be better contextualised (supporting the context-based and associative information management described in 2.2.2). Taking a step back to view this lack of metadata at a more conceptual level, leads us to the next insight:

What we can see in this section is that by paying attention to ecosystem information, metadata and provenance, we can open up a new space that, at the time of writing in 2022, almost no-one is building for. For people to manage their digital world, they need a map. This is the first step on the road to giving individuals the ability to have oversight of their personal data ecosystem and take action within it.

7.3.4 Obstacles to the HDR Objective of Ecosystem Negotiability

This section explains three distinct obstacles to ecosystem negotiability: the intristric structures that give data holders power, the trend of actively diminishing user agency, and the intractable data self.

7.3.4.1 Hegemony through Data Holding

It is in the pursuit of individual oversight [6.2.2] and decision-making involvement [6.2.3] that the impact of the power imbalance between data holders and individuals [2.1.2] becomes most clear; unlike the other HDR objectives, individuals cannot act to claim ecosystem negotiability for themselves. Negotiability means having the power to act, and in the context of systems and interfaces owned and designed by service providers that power can only be given. The hegemony of data holders is therefore is the greatest obstacle to this objective, so it is vital to examine the nature of that power - where does it come from?

A helpful analogy for the relationship between provider and user can be seen in the design of Jeremy Bentham’s Panopticon (Bentham and Bozovic, 2011), a real-world version of which is pictured in Figure X: an 18th century prison architecture design that would elevate the power of the (hidden) prison guards to observe all the prisoners easily at any time while removing prisoners’ privacy and providing no ability to observe those in power. As in Orwell’s Nineteen Eighty-Four, individuals are unable to know when they are being watched (in this case because the guards are hidden from view by one-way screens), thus are forced into compliance. Structuralist philosopher Foucault interpreted the Panopticon as a political design, recognising that human environments can be configured to influence or regulate behaviour, in order to defend the power of the ruling class (Foucault, 1975). Such designs embody his four principles:

• Pervasive Power: the guards see everything all the prisoners do, all the time
• Obscure Power: the guards can see into any cell at any time, but the prisoners can’t know when, how or why they are being observed
• Direct Violence Made Structural: the structure motivates the prisoners to self-regulate their behaviour without being coerced (through beating or punishment)
• Structural Violence Made Profitable: having been made compliant by the structure, the prisoners can be put to work for the benefit of those in power, as it is the only option available to them.

We can see at least three of these traits in modern Internet platforms such as Facebook today: these platforms monitor user behaviour (pervasive power) without their knowledge and without accountability (obscure power). Interfaces are designed to offer only those actions that benefit the platforms (for example, clicking ads, sharing content or spending more time on site (structural violence made profitable). This has happened through the processes of platformisation and infrastructurisation (Helmond, 2015; Plantin et al., 2018), which have supplanted the Web 2.0-era promise of a free, open Internet that could be a great leveller and empower individuals.

Through the control of the data and of the design of the interfaces through which the data is made available – the only channel through which they can be observed – service providers and platforms assert a structural power over the digital landscape. Just as the design of the panopticon regulates the behaviour of the prisoners, so the configuration of the platforms, apps and service interfaces we use regulate and limits the behaviour of users. As Lessig wrote, ‘Code is law.’ (Lessig, 2000). This infrastructural power is explained further in the insight below.

Looking deeper into theories of power reveals that structural power is not the only form of power which modern-day data-centric service providers hold. Jasperson et al.’s extensive review of types of power in the context of technology organisations (Jasperson et al., 2002) identifies 23 different power paradigms, of which at least 13 can be, and are, asserted by data-centric organisations today:

• authority: ownership of technology or infrastructure (for example of websites, servers and code)
• resource control: controlling the flow of resources (in this case of information/data)
• systems/structural power structural manipulation of others (as detailed above)
• rational power: controlling decision-making processes
• disciplinary power: using an influential position to affect others’ mental models (for example, positioning location tracking as theft resilience
• zero sum power: winning a battle for ownership/resource control at the other party’s expense (e.g. losing control of your sacrificed data)
• behavioural influence: persuading others to carry out the desired behaviour (e.g. restricting features to motivate subscription payments, or promoting certain content or actions)
• interpretative influence: determining how reality is externally represented (e.g. Facebook determining the way in which your social network is represented to you)
• network centrality: becoming an indispensable hub of a wider ecosystem (for example, Facebook/Google dominance in online ad-brokering)
• processual power: changing processes for competitive advantage (for example, platforms offering preferential APIs or rates to compliant partners)
• socially shaped power: influencing a wide audience to settle upon a preferred interpretation (e.g. using dominant market position to dominate debates e.g. about privacy norms)
• interpretive power: creating the internal representations of reality within an organisation (for example, presenting unpopular attitudes to data privacy to staff as normal/acceptable/beneficial for business)

INSIGHT 6: Data Holders use Four Levers of Infrastructural Power
Hestia.ai [7.1.1] have produced a model to explain the mechanisms by which powerful technology companies gain power and use it to shape today’s digital landscape. In this model, infrastructural power comes from three things: technical ability, organisational ability, and the acquisition of data about individuals and populations. Thus, as organisations (especially platforms) collect more data, and grow in market influence or technical capability, they gain power over individuals and over other organisations. They exert power in four quadrants, using four ‘levers’. Simplified and expressed in the terms of this thesis, these are:
1. Collect & Interpret Data to Acquire Knowledge: Data and signals are collected from individuals and interpreted in order to infer their intents and interests. For example, Google collects raw GPS and wi-fi hotspot data from mobile phones, which it then statistically analyses to infer which shops or venues you visited and what forms of transport you used, increasing Google’s knowledge about individuals and populations.
2. Present Content and Configure Structures to Influence Individual Behaviour: Knowledge of individual intents and interests is exploited within user interfaces to influence desired individual actions. For example, Facebook or presents a user with a product relevant to their interests, which they are motivated to click upon, generating ad revenue. Another example would be Twitter manipulating the content of the user’s feed to show more tweets from conversation topics where they can show promoted tweets, increasing ad revenue.
3. Configure Structures to Improve Knowledge Acquisition: A provider uses its dominant position to force other organisations to improve the provider’s ability to acquire knowledge. For example, Google provides free analytics tools to web developers, but requires the end users of those client websites to supply visitor data back to Google, increasing their ability to acquire knowledge about individuals and populations.
4. Configure Structures to Disadvantage Others: Certain providers (typically of operating systems or popular devices) can configure the structural relationships between other parties. For example, a smartphone manufacturer could limit data exchange between other apps, while still extensively collecting data signals themselves, such as when Google was found to be collecting call history from Android’s dialer app.
The precise mechanisms and techniques employed by platforms and providers when exerting their infrastructural powers as well as the social and market consequences of these practices are explored in detail in Hestia.ai’s digipower technical reports, of which I was a co-author (Bowyer, Pidoux, et al., 2022; Pidoux et al., 2022).
An important aspect highlighted in the research is that providers’ power is far greater than many realise: Unlike in the physical realm, providers of popular online platforms can reconfigure the landscape to change the way that individuals perceive reality, in line with the powers of interpretative influence, behavioural influence and socially shaped power described above (Bowyer, Pidoux, et al., 2022). Providers control the extent to which (if at all) the data stored behind the scenes, and the internal processes that use that data, are visible, and how such data and processes are represented.
The above model shows that the accumulation of data (and hence, information) is implicitly and objectively a form of power. This theory is consistent with participants’ observations in 5.4.4.1 that data holding and limiting access to it is a source of power. We can predict that as long as current platforms and service providers are free to collect so much personal information, the information landscape will remain imbalanced and individuals will not be able to acquire ecosystem negotiability.
Through this insight it is clear that the most powerful data holders exert huge influence over the digital landscape, in terms of what is know-able and what is do-able. Individuals or activists’ abilities to balance the landscape are hindered by the fact that they are operating in a landscape that the incumbent platform and service providers effectively control.

[TODO make this an inset box not a table]

The Intractable Data Self

The third and final obstacle I have identified to the objective of ecosystem negotiability is the intractable data self. As identified in the pilot study (Bowyer et al., 2018), and in Case Study Two [5.4.4.1] data about individuals serves as their proxy. It serves as their data self, and if it is incomplete, inaccurate or unfair, which is highly likely given the difficulties of representing people in data (Martin, 2007; Cornford, Baines and Wilson, 2013), this can cause harm (Bowyer et al., 2018) or undermine attempts to help individuals (Cornford, Baines and Wilson, 2013). Yet currently, although some legal rights to data correction exist (Information Commissioner’s Office, 2018), people lack practical abilities to modify or assert control over the most important version of themselves as far as providers are concerned: the version of them that exists in data. Even when data can be seen (such as via a support worker or GDPR data access requests) people lack the ability to exert influence over their data self [5.5.2; Cornford, Baines and Wilson (2013)]. To address this obstacle, the most likely direction would be to explore possibilities by which people could take a role in the curation of their data self, as both Case Studies have proposed [4.4.3; 5.5.2] and 6.3 have proposed.

To conclude consideration of this objective, it is noteworthy that to date, research and innovation on personal data ecosystem negotiability has been very limited. It is much easier to find business models and research funding for specific, well-defined contexts. Due to the lack of business incentive, only non-profit socially-focussed research organisations such as BBC R&D and Sitra have found themselves well-equipped to explore this problem space. Nonetheless, despite these challenges, there is an urgent societal need for researchers, designers, policymakers and innovators to explore how trends of diminishing agency can be reverse to involve people with their data. People need to be reconnected with their data selves, and given control over their digital lives, at the broadest level, rather than being excluded.

7.3.5 Obstacles to the HDR Objective of Effective, Commercially Viable and Desirable Systems

In the previous four subsections, the obstacles to the four identified HDR objectives [7.2.4] were considered. However, during attempts to tackle these objectives, and through observation of how the public and businesses were engaging with the growing Personal Data Economy, it became clear that there are certain obstacles that are specifically faced in this sector that affect all efforts to make progress towards improving HDR. The main challenge is around building such disruptive systems that are so different from the status quo: businesses and individuals will not readily invest time and money in HDR, because it is unfamiliar. Customers are not demanding HDR capabilities in their lives, and, all but the most socially responsible businesses do not immediately see the value in something that runs so contrary to their current business models which are based on the accumulation of data and the control of customer experiences.

Today, data is overwhelming, complex, and ‘sounds boring’. There is no denying that currently, engaging with one’s personal data economy to any degree more than that of passive consumer, is hard work. People routinely accept data sacrifice, click through T&Cs and cookie banners and are unwilling (or in some cases lack sufficient technical literacy, comprehension or skill) to do the work of asserting control over their digital lives. There is not a clear demand for holistic and novel ways of managing your digital life and exerting agency and negotiability over it. Across both Case Studies and the PDS work at BBC R&D, it was clear that even if new human-centric information systems and more inclusive service interaction practices could be created, we cannot assume that people will be inclined to use them in great numbers, especially not if it seems like hard work or not worthwhile. This can be seen as an obstacle that affects all HDR improvement approaches we see, and indeed is why many companies in the emergent PDE economy [2.3.4) struggle to find a business model - while there are clear benefits, better HDR does not appear to something that a mainstream audience would be directly be willing to pay for. But this should not deter disruptive innovation nor does it indicate that such offerings would not be useful. As automobile pioneer Henry Ford famously said, “If I had asked people what they wanted, they would have said faster horses.” Nonetheless, it is a clear overarching obstacle to overcome.

[TODO make this inset box not a table, images appear in box]

The kind of life-spanning, unifying interfaces described in the insight above are nothing like the interfaces that are built today, as they span across different providers’ data and services. This highlights the secondary obstacle that all HDR system builders will face, whichever objective they wish to target: closed, self-interested organisations with a lack of interoperability. Building an HDR system will necessarily involve connecting to systems of different providers that have different touchpoints into an individual’s life and world. Yet most companies act in closed, introspective and non-cooperative ways to further their own interest. Companies like Apple, Amazon, Microsoft, Facebook and Google (the so-called ‘big five’) build proprietary, incompatible silos or ‘walled gardens’ – sub-Internets that pretend that the alternatives do not even exist, in order to encourage a flow of money and attention to their own products and services. Commercial motives encourage them to get users to spend time in their own proprietary spaces (so that resultant ad revenue can be captured) and in order to maintain subscription revenues it is in providers’ interests to make it hard for individuals to leave or switch providers. In effect, providers build for a world that does not exist, where every individual is imagined to only interact with that single company’s interfaces. I would argue, for example, that Google’s venture into social networking with Google+ did not succeed because it failed to build for a reality where most people and their friends were already on Facebook. But one can understand their motives; there is little incentive to open up the ecosystem when the free flow of information and of users might result in loss of income for the company in question. Users with negotiability would be more able to leave. And this also encourages keeping users in the dark [5.4.2]. The less agency and negotiability that users have, the more freedom the provider has to do exactly what they want with their data. In this context, users are ‘pathetic dots’ (Lessig, 2000) or ‘docile bodies’ (Foucault, 1975).

The tendency of organisations to work in closed, introspective practices and to be resistant to opening up data or services is not solely motivated by commercial reasons: the public sector has a vastly complex, closed and fragmented ecosystem [Pollock (2011); Copeland (2015); 4.1.2]. Efforts to build a system to share health data with support workers for the SILVER project [7.1.1] proved hugely challenging. Sometimes the challenge was a more technical one - incompatiable data formats that are hard to reconcile, or data being stored in legacy systems with no public API that would allow programmatic access to that data, or issues around licensing. Data sharing agreements must be established, especially in the public sector which is by its nature more liable to scrutiny and accountability. But more than these technical or procedural issues there was resistance to change in data processes and an unwillingness to share data between agencies, often motivated by a fear of legal repercussions. Data-centricism encourages insular thinking: it encourages organisations to codify the world into their own systems, processes and formats for their own use.

And yet, for effective HDR, data needs to be separable from services. The more users data is tightly coupled to specific services, the less agency users have and the harder it is to build life-centric systems. On BBC R&D’s Cornmarket project, attempts to build an interface for users to import data from multiple popular Internet services proved to be a hugely complicated endeavour, requiring access to many different APIs or manual exports and imports of data by users. There needs to be greater interoperability and greater establishment and adoption of standard formats for exchanging human information (as distinct from establishing standards for data or service-specific APIs). As mentioned above, platformisation breaks the Open Web (Plantin et al., 2018). To overcome this, companies must be persuaded that human-centric thinking, interoperability and transparency has not just social benefits, but business benefits too.

But at an abstract level the technical obstacle, the problem is one that has always faced the tech industry, which is that there often is no universally agreed way to represent important concepts - in this case human-centric information concepts such as events, social media posts, website visits, location history information, app activity, etc. And any entity that does create a standard then faces the challenge of trying to persuade others that their standard is the best one to use. In general, standards work best when established by non-commercial industrial standards bodies (for example the World Wide Web Consortium (W3C) or International Organisation for Standardization (ISO) and then mandated through policy such as European Union law. Such standards much be established with input from industry experts.

[TODO inset box not table, including image in box]

Machine learning technologies and Artificial Intelligence have pushed machine understanding of human words, images and content to impressive levels in recent years and such technologies can certainly be helpful, but in fact at the core what we are talking about here is somemthing much simpler than AI; It is simply about labelling datapoints in as many different ways as possible so that those datapoints can be associatively retrieved from many different angles, and providing humans with ways to amend incorrect labels and to reclassify data or apply new semantic associations. Issues of interoperability for PDS systems are being actively explored and developed in the ‘Solid’ community (Bansal, 2018; Berners-Lee, 2022) in pursuit of a decentralised web (Verborgh, 2017).

Such approaches are in their infancy, and have not yet been adopted extensively in commercial settings. Even after addressing the obstacles of end-user buy-in and the technical complexities of building human-centric systems, data-driven corporations, motivated as they are by profit and business success (and smaller online organisations too) need to be persuaded of the business value of transparency, interoperability and human-centricity. This is explored further in 7.4.5.

In summary, whichever of the above four HDR objectives are targeted, all HDR reformers involved in building HDR systems must:

1. create, adopt and co-ordinate around new standards for human information storage and management
2. invest in systems that elevate computers from data-processing machines to human-information-processing machines, and
3. make a persuasive case to both businesses and individuals that the new approach offers tangible, previously unavailable value.

7.4.1 A Primer on Theories of Change (ToC)

To provide a structure for cataloguing the insights conveyed by this section, I use a Theory of Change (ToC) framing. ToC is a set of methodologies is commonly used by philanthropists, educators and those trying to improve the lives of disadvantaged populations (Brest, 2010); the theories can be used in different ways including planning, participatory design and field evaluation of the effectiveness of new initiatives. There are many different implementations, but common to most of them is a focus on explicitly mapping out desired outcomes (Taplin and Clark, 2012) with a clear focus on who is acting and whether the change being brought about is a change in action, or a change in thinking (Es, Guijt and Vogel, 2015). In this section, ToC theory will be used in a very limited way, not as a methodology but simply to provide a structural frame for proposed changes, as described below. Using ToC to perform evaluation of the effectiveness of proposed change approaches in action in society would be well beyond the scope of this thesis. Nonetheless, this frame is a useful way to map out the different approaches to changing the world in pursuit of the ideal of better HDR.

Figure 29 illustrates the aspects of ToC thinking that are important when using this frame. Specifically, desired changes can be broken down into:

• Internal changes: changes in thinking, feeling, reasoning, understanding, attitudes or identity.
• External changes: changes in actions, behaviour, interactions, structure, policy, technological capability, processes and the external environment.

At the same time, desired changes can be broken down into:

• Individual changes: changes to individual thought or actions
• Collective changes: changes to the thoughts or actions of groups of people together, or to the systems, practices and norms of society at large.

These two splits produce four dimensions of change, and form four quadrants representing different types of change, which are shown in Figure 29 and described here:

• Individual/Internal (II): This top-left quadrant represents changes to what individuals know and understand, and to how they think, feel and plan to take action.
• Individual/External (IE): This top-right quadrant represents changes to how individuals’ relationships with others; acting (or being enabled to act) differently in their daily lives and when interacting within society.
• Collective/Internal (CI): This bottom-left quadrant represents changes in the shared knowledge of groups of people or to the collective identity or values of social groups.
• Collective/External (CE): This bottom-right quadrant represents changes to the structures and procedures within which people operate, including technology, law, societal norms and communications.

Key to ToC thinking is the idea that making changes in one quadrant can stimulate change in others; for example, collective learning about data attitudes and practices, such as the research conducted in this PhD, (lower left quadrant) could inform the design of new technologies, interfaces or processes (lower right quadrant), which if built could make new structures available to have an impact on improving individual-provider relationships (upper-right quadrant). The changes to those relationships could then in turn lead to individuals thinking and feeling differently (upper left quadrant), for example feeling more empowered or having greater awareness of data practices. THe diagram at the start of each of the following four subsections illustrate that approach’s trajectory of change through the ToC quadrants.

7.4.2 Approach 1 to Improving HDR: Discovery-Driven Activism

[DIAGRAM PLAN - TODO ADD DIAGRAM] [individuals informing/powering collectives Collectives helping individuals Using Data to Demand Change in Practice => which in turn enables individuals with stronger capabilities and better transparency & insights also data access & understanding services helping collectives and individuals info pulling from the bottom right and top left into bottom left, then pushing up into top two.]

The approach to HDR reform presented in this section embraces the activist aspect of being a recursive public [7.2.5], as well as the idea of reconfiguring one’s world. This approach focuses on the realities of the current data-centric provider ecosystem, and is focused upon deeply understanding it so that it can be challenged from a grounded position of strength. The approach, which would be applied typically to a single service provider, app or platform (which could also be a public sector service) is fourfold:

1. Gather Evidence: Using all available means, gather personal data and information from the target organisation.
2. Determine Current Practice: Analyse the gathered data and information, using targeted approaches to dig deeper where needed, to establish facts about how the target organisation stores and handles personal data.
3. Identify Desired Change: By consideration of the specific practices observed from the analysis, specific problematic practices, gaps or areas where improvement or change is needed are identified.
4. Appeal for, or Compel, Others to Act: Using all available means, directly or indirectly persuade or force the target organisation to execute the desired change.

I describe this approach as Discovery-Driven Activism. The discovery phase, which aims simply to establish facts about the past or current data practices of the target organisation, can be broad (‘let’s see what we find’, as in the digipower investigation [7.1.1]), or highly targeted, such as when The Citizens (a non-profit pressure group in the UK who ‘use impact journalism to hold government and big tech to account’ (‘The citizens - about us’, 2020)) used Subject Access Requests to investigate a breach of personal data by the Labour Party in the UK (Colbert, 2022).

Subject Access Requests and Data Portability Requests (Information Commissioner’s Office, 2018) are two of the most powerful tools for this kind of investigation. Similarly to Freedom of Information Requests which have used to obtain otherwise hidden data and information from governments and public sector organisations (BBC News, 2014), so these new data access rights are beginning to be used to force commercial organisations to release personal data or information about data processing. There are challenges in non-compliance, as discussed in 5.4.2.2 and 5.5.1, but the ability for the individual to ask very broad or very precisely targeted questions and to be able to threaten a complaint to a Data Protection Authority if that question is not answered, is a significant new power that HDR reformers can exploit.

Facebook’s DYI tool, mentioned in the insight above, represents a useful class of tool in the arsenal of the activist HDR reformer. Along with Google Takeout, it is one of number of ‘data download portals’ that allow users to download their own data. Since GDPR’s introduction in 2018, an increasing number of large online platforms including Facebook, Google, Apple, Netflix, Twitter, Spotify, Uber, Instagram and Strava, faced with the need to reduce the cost impact of GDPR request handling for their large userbases, have developed and augmented online self-service portals available where users can download a copy of their personal data. This has some advantages over Subject Access Requests in that data can usually be obtained within minutes or hours rather than taking up to 30 days, but has some disadvantages in that the data returned is a voluntary offering by the company, that may not cover the data that the individual is seeking and does not provide any ability to ask follow-up questions. This technique was sometimes used as a fallback means to obtain data in Case Study Two, and was used more strategically in the digipower project, where its merits and limitations are discussed [7.1.1; Bowyer, Pidoux, et al. (2022)]

Both access requests and download portals rely on the organisation in question to be transparent, accurate and thorough in their provision of information, but an alternative technique of data flow auditing allows individuals to investigate and collect data on the actual behaviour of a target organisation. This was used effectively in the digipower project [7.1.1]. Using an Android app called TrackerControl (Kollnig, 2021), a service provider’s app can be monitored while the user is using it normally, to see which servers or domains that app is contacting (and one can imply, exchanging data with). Apple has recently introduced an equivalent function on iOS known as App Activity Reports (Apple, 2022), providing iPhone users with the same ability as part of the phone’s operating system. This has limitations in that the content of the data exchanges is not known, but can serve as a valuable tool to verify claims made in privacy policies or GDPR responses, and also as a means to generate questions for further investigation, for example by identifying third parties such as data brokers which the target organisation may be sharing personal data with. This technique is described further in (Bowyer, Pidoux, et al., 2022), along with a comparison of the different techniques of data flow auditing, data download portals and data access requests.

In general, what the discovery-driven activism approach highlights is that there is a role for pro-active citizens to play in challenging the power of data-holding organisations by treating those organisations as a subject of investigation, both in research (Walby and Larsen, 2012) and in the pursuit of improving civic society (Schrock, 2016).

Once information has been obtained, the HDR reformer activist can use a variety of means to try to bring about the desired change. If a target organisation fails to comply with a data access request, or a demand to erase or correct data, they can be reported to the Data Protection Authority. In some cases even the threat of this (which can carry a large fine) can be enough to compel the organisation to change. If a breach of law is found, the target organisation could be taken to court, as seen in the Schrems case above, which resulted in new legislation that Facebook had to comply with (Kuchler, 2018). As well as individual cases, this also often happens in the form of class action lawsuits, as with Facebook and Cambridge Analytica (Bowcott and Hern, 2018). Increasingly, unethical or illegal data practices are being challenged. In some cases, such extreme measures are not needed. Simply making data available to the public can be empowering to society at large. This approach has been demonstrated by the UK website TheyWorkForYou, which increases democratic accountability of MPs by making MP’s votes and public statements more readily accessible (mySociety, 2004). As well as structured ‘impact journalism’ such as that conducted by The Citizens as mentioned above, another technique available to individual activists is public shaming of misbehaving organisations, especially on Twitter. While the ethics of this are complex and it does not always succeed, the technique has been used effectively to force organisations to change, in order that they might avoid further bad publicity (Silver, 2014; Braw, 2022).

Having identified that there is a clear trajectory where individuals and collectives can obtain data to empower them, it is clear that this complex work can be supported. We see the emergence of ‘data access & understanding services’, with entrepreneurs and activist enthusiasts:

• creating tools to help people get their data back from organisations, such as Tap My Data (‘Tap my data’, 2022), AccessMyInfo (‘Access my info’, 2022) and My Data Done Right (‘Privacy salon & my data done right’, 2022);
• creating tools to help people understand their personal data, such as Ethi (Jelly, 2021) and digipower.academy (‘About digipower.academy’, 2022); and
• helping collectives and journalists with training, workshops and skills development to take advantage of their data rights and understand data ecosystems, like Hestia.ai (Dehaye, 2019).

If such emergent endeavours can be supported and enabled to flourish, that could make HDR reformers using the discovery-driven activism approach more successful by ensuring that a lack of legal, technical or investigative skill does not become a barrier to any HDR reformer wanting to use this approach.

This approach shows that there is a role for independent actors and organisations to carry out discovery-driven activism - access requests, complaints, legal challenges, public campaigns and more. Discovery-driven activism can empower individuals and collectives to incrementally work towards building the world of better HDR that this thesis outlines.

7.4.3 Approach 2 to Improving HDR: Building the Human-centric Future

[TODO diagram FRAME AS DIAGRAM Building new designs (reaching into understanding, LITREF data enabled design and Human values) Delivering new structural capabilities. Enabling new individual and collective perspectives.]

The approach to HDR reform presented in this section focuses on the gaps in individual data interaction capability that exist today. The objective here is to design and build proofs of concept for novel human-centric information systems that can deliver people new capabilities over their data. In this approach, the focus is more introspective than Approach 1: it is about how the individual can improve their relationship with data in the context of their own digital life. The bulk of this section describes specific design ideas developed by myself and colleagues at BBC R&D during my 2020-2021 research internship on the Cornmarket project. As established in Insight 2, one of the most promising models for giving people a new and improved relationship with their data is to create a place where one’s scattered (Abiteboul, André and Kaplan, 2015) personal data can be stored and aggregated in one place (Jones, 2011), and based on Insight 3, that place should offer individuals the means to use their life and ecosystem information as a material they can explore, examine or repurpose as they see fit. This leads to the vision of a Personal Data Store [2.3.4] or into which one can unify the data from the different parts of your digital life (as depicted in the AllOfMe vision video (‘AllofMe.com Teaser Clip’, 2008)), and meet public demand for “control over your data” (Teevan, 2001; Hartman et al., 2020). The Cornmarket R&D project therefore sought to develop a human-centred (ie. non-commercial) PDS proof of concept, as conceptualised in Figure X.

7.4.3.1 Designing a Life Interface: Transforming Data into Life Information

The first challenge in designing such as a system is to consider what data will be stored, and how it might be represented. At a high level, I identified a number of different types of data that a user might wish to store in a PDS:

Then, considering the earlier observation by myself (Bowyer, 2011) and proponents of temporal PIM systems [2.2.2] that time can be used as a unifying concept for personal information, I considered how we might represent occurrences within personal data as happenings, anchored against a particular point in time. In effect, this would focus on different properties of the data, much like Karger’s lenses [Karger et al. (2005); #2.2.2-contextual].

In modelling data as life information using the concepts shown in Figure X [TODO link to Life Concepts diagram] and Figure X [TODO link to Happenings above], it is important to come up with as simple a model as possible, so that the life interface is not overwhelmingly complex and unmanageable. Over several iterations, I was able to reduce the modelling of life information in a PDS down to the following four types - activities, interactions, transactions and datapoints, which could correspond to four views or lenses within the user interface:

Having decided upon models for the information within a PDS, it becomes much easier to design visual presentations of that information. Figure X shows a design mockup of some of the views of life information that one could offer within a PDS:

7.4.3.2 Life Partitioning

What has been shown in Figure X is the beginnings of imagining what a life interface might look like, a single holistic interface covering all aspects of one’s digital life.

Previous HCI design work [2.3.1] has focused on, in the first wave, the functionality of the machine; in the second wave, on the common tasks of a work environment; and in the third wave, on classes of users and the commonalities of experiences in everyday life. But the challenge designers of life interfaces face is even more difficult. As Lindley noted, no single arrangement of information will suffice because in the same context two people may have different needs [Lindley et al. (2018); #2.2.2-contextual]. Because of the subjectivity principles [Bergman, Beyth-Marom and Nachmias (2003); #2.2.2-subjective], we are now faced with the need to build an interface that is suited to the individual, even though every individual has different needs. The design requirements for the interface are likely unique to the individual. Therefore, any life interface design must be able support different mental models - and as the life sketching exercise [Figure 10] in Case Study Two showed, people have many different ways of compartmentalising their life.

During a life interface design hack week at BBC R&D, colleague Jasmine Cox and I explored this problem. We considered that a key element of the interface design would need to be the versatility to partition one’s life in different ways, and then to be able to assign different collections of life information to the different partitions. As a visual illustration of one possible mental model for life partitioning, consider Figure X, where, based on an idea I had had years earlier, I modified a Cluedo™ board to represent the ‘rooms of your digital life’:

We also imagined other mental models, such as partitioning your life according to parts of the body (heart for relationships, body for fitness, brain for current projects, etc.) or a landscape with forest, lake and buildings representing different things.

Whatever the visual metaphor engaged, the important thing is the functionality - being able to use these partitions to filter [2.1.4] your life information and focus on a particular perspective [Lansdale and Edmonds (1992); Krishnan and Jones (2005); 2.2.2]. Deciding to focus on a particular aspect of one’s life is analogous to fixing a conceptual anchor [Teevan (2001); 2.2.2].

We then produced a mock app workflow for assigning different elements of your life (such as people, places or topics) to different partitions of your life, and viewing a visualisation of those different partitions in some subjectively meaningful form before filtering on a particular life partition and then being presented with a timeline [2.2.2] of life information pieces associated with that part of life. This is illustrated in Figure X, with artwork by Jasmine Cox:

7.4.3.3 Entity Extraction from Pooled Life Data

INSIGHT 11: Automating the Identification of Entities can enhance Machine Understanding and Unburden Life Interface Users.

Having identified the need to assign every piece of a user’s life information to a particular partition (or multiple partitions) of their life, it quickly becomes apparent that this would be too much work for the user to do alone. Systems that use manual categorisation and tagging to classify information work best with a large userbase to contribute effort to the classification operation (Golder and Huberman, 2006). As part of the explorations of PDS approaches at BBC R&D I therefore also examined how this challenge might be addressed (considering also that having a lot of effort could be a deterrent to adoption [Objective 5]). I identified an approach that could help with this problem: If the entities (for example, a person, a place, an event or a topic) associated with a piece of data can be identified, then a lot of the assignment of data to life partitions can be handled automatically. For example, association with your office location would indicate that any data associated to that location is likely to relate to the ‘work’ part of your life, and this could be done automatically, reducing the effort for the PDS user. The process of identifying entities within data, known as entity extraction or named entity recognition (NER) is a well-established technique, which relies on the trained recognition of proper nouns and keywords combined with the statistical analysis of sentence grammar (Marshall, 2019). This technique is used extensively in text-mining products within the Content Analytics industry such as those produced by my former employer, OpenText (‘What is text mining and content analytics?’, 2022). However, in the context of a PDS, I propose that new techniques can be applied, making use of the data touchpoints into different parts of an individual’s life to identify entities relevant to them personally (including, for example, names of friends or private projects that a standard NER solution would not detect). Data is full of references to entities that have personal relevance in your life. Finding these allows meaningful metadata to be attached to each datapoint. Figure X shows how a large number of entities could be detected from different parts of an individual’s data once it has been imported into a PDS environment:

(continues…)

This sort of approach could be quite powerful in reducing the effort for life interface users. By scanning the data, the most prevalent entities could be identified, and the user need only assign the entities to different parts of their life, as illustrated in the first two frames of Figure X [REF to mockup]. This would then allow hundreds of associated data points which had been programmatically associated to that entity, to be assigned to the correct ‘bucket’ or life partition. While such an approach would not be perfect, and there would need to be some corrections made by the user, this is far preferable to them having to provide all the classifications and is likely to motivate greater engagement. I have observed in user experience design and consideration of productivity systems that users are more motivated to correct errors, than to fill in a blank page. Philosophically, we are moving here towards a learning system, a system that can be told when it is right and when it is wrong, and get better at classifying things correctly, analogous to the way an executive might train an assistant to anticipate his/her needs better, a sort of digital life assistant [ADD REF]. Bayesian classification techniques could also be used to help with the learning here (Authors, 2022b). . This approach is also useful for ecosystem detection, as outlined in Insight 4 - as identification of relationships with external entities is a key first step to mapping a user’s ecosystem.

Insight 11 offers a practical, theory-informed approach as to how we might start to build systems that are more able to understand the meaning of human information [Insight 8]. This is a technique that is already being used by corporations. For example, Facebook has developed an internal system called world2vec, shown in Figure X, through which it mines information about the world from the data that people generate through social media posts, messages and interactions. This information is exploited for commercial benefit. It is about time similar techniques were used for individual benefit and personal data empowerment.

Automatic ‘best guess’ attempts to arrange personal data in a PDS can go further than just association of related entities. In many cases, it is possible to programmatically determine the nature of a piece of data. In Figure X[examine a piece] is shown a detailed approach for identifying and classifying data automatically in a PDS, drawing on the previously identified data types in Figure X[types], the data attributes in Figure X[attributes], and the entities identified in Figure X[entities]. In Figure X[determining nature], a more abstracted approach is shown looking at how data could be classified according to the four different life information concepts in Figure X[4 ways PDS].

7.4.3.4 Designing Capabilities for the Manipulation of Life Information

Moving beyond the initial challenge of classifying and arranging human information within a life interface, a core consideration in designing such an interface is to consider what capabilities the user might be given over the pieces of life information represented in the interface. It is through the provision of a wide range of useful operations upon information items that the information will start to feel like a material as described in Insight 3 and deliver the new capabilities needed for success Insight 7.

There are two ways to consider this problem - one starting with the data, and another starting with the user.

Starting with the data, and considering the different types of data that a user might be able to gather from a across their digital life, I identified that each datapoint or dataset is likely to have properties which determine what can be done with it. A number of possibilities are shown in Figure X:

These attributes could then be used to determine which datapoints would be eligible for inclusion in different views (or lenses (Karger et al., 2005)) of the data, such as tables, graphs, maps, lists, media thumbnail views, etc.

If we consider capabilities from the user’s perspective, it is useful to recognise that visualisations alone (such as those in [REF FigureX-dashboard] are not sufficient to meet the need for either interrogability or malleability [7.3.2]. To address the need for malleability, we need to determine what actions an individual might want to perform on a piece of life information. To explore this design question, we can draw on multiple sources of inspiration:

• actions identified in prior PIM research & design work [2.2.2] such as arranging, browsing, keeping, and associating;
• actions needed for SI [2.2.3] such as combining, history viewing, interpreting patterns;
• actions needed for effective access [2.1.4] such as filtering, zooming, getting detail on demand; and
• actions inspired by the eight lenses of how people think of data [Table 15] such as summarising, deriving or remixing.

Having developed a large set of initial design ideas for possible actions, I worked with my BBC colleague Chris Gameson to distil this information design thinking into a 12 data card designs that could encapsulate some of the most common actions a user might want to perform upon personal data in a PDS. These are shown in Figure X. These had a similar purpose to the cards in my pilot study [1.3.1] and Case Study One [4.2.1], acting as ‘things to think with’ (Bowyer et al., 2018).

As identified in chapter 5 [5.3.3; 5.4.2.3] and explored further in 6.1.3, being able to ask questions of data is important to individuals. Adddressing this need for interrogability in a similar approach, we produced 10 data card designs to encapsulate some of the different common questions that users might want to ask of their data, as shown in Figure X.

These actions and questions act as a design building blocks, a precursor that can inform the design of features for a life interface that could provide the user with new capabilities to interrogate and manipulate the data in a much more interactive way.

Once information has been correctly identified semantically (using approaches such as those in Figure X[naturedetermining] and Figure X[identifingattributes]), and associated to entities, it then becomes possible to enrich users’ search and browse capabilities by enabling the use of facets {Televiciute (2020)} to allow users to precisely target a set of data points or a particular data item. An example of how this could be done is shown in Figure X, another design artefact from my work at BBC R&D. This is not a user interface mockup but rather a three-stage strategy, with examples, for the different types of facets that one could offer a user to help them explore their data in powerful ways:

7.4.4 Approach 3 to Improving HDR: Defending Autonomy and Nurturing the Information Landscape

[TODO diagram: taking external protective action as collectives, surfacing, challenging, pushing for better enforcement of existing regulation]

While Approach 2 focuses on designing a better future, and Approach 1 focuses on identifying and campaigning for positive changes in provider practices, there is a need for a more direct approach to tackle an urgent and evident problem head on: the active diminishing of users’ agency [7.3.4.2]. In pursuit of mass market appeal and cost reduction, there is a trend towards the dumbing down of technology, which invariably leads to fewer features and less agency. Increasingly, companies encourage users to think of technology as a black box, which you cannot and should not look inside, as illustrated in Figure X, and discourage users from varying their usage, digging deeper or challenging the normal way of using a product:

This disempowering trend is seen across hardware, software and service industries, where providers attempt to restrict the ways in which technology can be used even after it has been purchased - cars that can only be serviced by official dealers, printers that can only use officially branded ink cartridges, smartphones that cannot be repaired without special tools and warranty invalidation, media files that cannot be copied or shared, online content that cannot be accessed in certain geographies, operating systems that block or discourage certain software from running, and banking applications that forbid screenshots. Apple, for example, encourages users to consider technology as ‘magical’, rather than as tools to be harnessed and understood. This approach is highly problematic for user agency:

“Magical design prioritises pleasing and surprising a passive user who can only use the solution as authorised.” – Cristiano Storni (Storni, 2014)

In order to challenge this trend towards disempowered, compliant users and protect individual autonomy and our ability to make sound judgements and hold organisations to account, direct activism and grassroots civic action is called for. This forms the core objective of this approach: to identify ways in which agency is being diminished, and to find ways to circumvent the new barriers in order to maintain existing levels of capability.

It is bad enough when a particular technology blocks you from doing what you want to do, but the trend is even worse when viewed at a broader level: As the digipower investigation highlighted [Insight 6], the information landscape that we inhabit as digital citizens is being manipulated to change what we see, what we understand of the world, and what we can do, in order to manipulate our behaviour, which can have harmful effects on democracy, agency and individual autonomy. Therefore a further goal here is to protect our ability to experience unbiased and unmanipulated information landscapes. A recent US Congress investigation shows how the big five use their dominant positions as device manufacturers, search engines or major Internet platforms to promote their own interests (Sisco, 2022). In a paper looking at the growing manipulation of search results, Shah and Bender consider how this landscape could be protected and accountability preserved. They describe their approach as establishing ‘guardrails for the status quo’, and I find this phrase to be an excellent summary of what Approach 3 is trying to do (Shah and Bender, 2022). We can also take this idea further through the idea of adversarial design (DiSalvo, 2012) which advocated using design processes to challenge the status quo. To understand how to gain traction here and carry out ‘design after design’ (Storni, 2014), it is important to understand the concept of seams, how they can be exploited, and the risks of losing them. This concept is explained in Insight 12.

[TODO make this an inset box not a table]

The reason that seams are so important, is that because they a point of friction, a point of resistance to the idea that data should be separable from services [Objective 5]. One of the goals of HDR, and particularly this approach, is to wrest control of data away from service providers, so that it can be truly free-flowing, as I envision in (Bowyer, 2018). In this approach, I have identified some of the key avenues for progressing this goal - through web extensions, accessibility tags, API exploitation and reverse engineering. But in the face of such extreme power, it is clear that such grassroots advocacy will not be enough. For example, web extensions are a powerful approach but are limited in that only those ‘power users’ who install the extension will gain the benefits. They do not directly help the layperson with their diminishing agency. Therefore, as outlined in [5.5.1], HDR reformers must also work to educate and persuade policymakers of the need for change. As an example of this, in my Case Study Two paper (Bowyer, Holt, et al., 2022) I called for better guidelines from the EU towards data holders. The European Data Protection Board launched a public consultation on a new set of guidelines they had constructed around GDPR (European Data Protection Board, 2022) and I contributed a detailed set of recommendations to that consultation, emphasising some of the important insights I have identified in this thesis, including a focus on human information, delivering ongoing understanding, establishing standards, and viewing data as an ongoing flow over time (Bowyer, 2022). This sort of engagement by HDR reformers is critical to help shape future policies and guidance, contributing expertise to help the civil servants making those rules. As HDR reformers we must continue to push for better regulation - as it really is the only force that can significantly change data-holding organisations’ practices. Better regulation is needed not only to help individual user autonomy, but to combat online extremism (Arthur, 2017). In this context it is important to note that the European Union is, at the time of writing in July 2022, developing a wide range of new laws that could help to improve the agency of individuals and the integrity of the information landscape:

• The Data Act (Board, 2022)
• The Data Governance Act (Commission, 2022a)
• The Digital Markets Act (Commission, 2022b)
• The Digital Services Act (Commission, 2022c)
• The Artificial Intelligence Act (Brakel, Uuk and Boine, 2022)

It is too early to evaluate exactly what these laws will change or what their impact will be, but HDR reformers should observe carefully: they will almost certainly play as pivotal a role in protecting the information landscape as the GDPR has in opening up data access to individuals.

In this approach, I have highlighted how important it is for HDR reformers to seize and harness the powers we are given, and fight to hold onto them. Groups of HDR reformers can combine development skills, innovation and disruptive design approaches to find and publicise new ways to circumvent providers’ efforts to control and limit users’ agency, and policymakers can prevent further erosion of individual agency through legislation and enforcement over the information landscape. As the popularised adage based on the words of John Philpot Curran and Thomas Jefferson goes, “The price of freedom is eternal vigilance”.

7.4.5 Approach 4 to Improving HDR: Winning Hearts and Minds: Teaching, Championing and Selling the Vision

[TODO diagram: Structural work in upper right - standards. Selling work in top level - show value to individuals. Selling work in top level - show value to organisations. Structural work in bottom right - systems. Individual work in top left - empower and educate individuals. all leading to new action of individuals in top right]

Approach 1, the investigative activist approach [7.4.2], and approach 3, the digital freedom fighter approach [7.4.4], can help with users’ ecosystem understanding [Objective 3] and ecosystem negotiability [Objective 4]. Meanwhile, Approach 2, the disruptive interaction designer approach, can help with user’s direct understanding [Objective 1] and capabilities [Objective 2] over their data. But as outlined in [Objective 5], these approaches will not have sufficient impact until and unless the goals of HDR reform become widely accepted among business leaders, citizens, journalists and politicians. And for that to be truly accepted, better HDR must be proven to work.

The call for better Human Data Relations is a call for a radical reconfiguration of today’s data world. As [Objective 5] outlines, where new systems are needed, system builders must invest in and see the value (not just to individuals but to their organisations) of HDR ideals. Where new policies are needed, politicians must be persuaded that HDR’s ideals are worthwhile and have public support. And most importantly for any change to occur, there must be a demand for change, and an engagement and appreciation of new HDR approaches once they become available.

Therefore, there is a complementary parallel trajectory of HDR reformer effort that is needed if the disruptive potential of HDR is to be realised. Across society, we must find ways to demonstrate, persuade and prove the value of better HDR. Collectively we could call this ‘motivational work’ in support of HDR reform. I have identified three aspects to this motivational work, which will be explored in the following subsections:

1. Education & Data Literacy [7.4.5.1]
2. Demonstrating Business Value [7.4.5.2]
3. Proving the Viability [7.4.5.3]